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(57) Abstract: A computer 

assisted orthopedic surgery planner 
software for generation of 3D (three 
dimensional) solid bone models from 
two or more 2D (two dimensional) 
X-ray images of a patient's bone. The 
computer assisted orthopedic surgery 
planner software reconstructs the 
bone contours by starting with a 3D 
template bone and deforming the 3D 
template bone to substantially match 
the geometry of the patient's bone. A 
surgical planner and simulator module 
of the computer assisted orthopedic 
surgery planner software generates a 
simulated surgery plan showing the 
animation of the bone distraction 
process, the type and the size of the 
fixator frame to be mounted on the 
patient's bone, the frame mounting 
plan, the osteotomy/coricotomy site 
location and the day -by -day length 

adjustment schedule for each fixator sunt. All bone models and surgery plans are shown as 3D graphics on a computer screen to 
provide realistic, pre-surgery guidance to the surgeon. Post-operative surgical data may be fed back into the computer assisted 
orthopedic surgery planner software to revise the earlier specified bone distraction trajectory in view of any discrepancy between 
the pre -operative plan data and the actual, post-operative data. 
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COMPUTER-AIDED BONE DISTRACTION 

5 

Inventors: Norman Krause, Robert Mendicino, Kenji Shimada, Lee Weiss, Takeo Kanade 

CROSS REFERENCE TO RELATED APPLICATIONS 
10 (Not Applicable) 

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR 
DEVELOPMENT 
15 (Not Applicable) 

BACKGROUND OF THE INVENTION 

20 Field of the Invention 

The present invention broadly relates to the field of orthopedic surgery, and more 
particularly, to computer assisted orthopedic surgery that uses two or more X-ray images of a 
patient's bone to generate a computer-based 3D (three dimensional) model of the patient's 
bone and a computer-based surgical plan for the doctor. 

25 

Description of the Related Art 

Bone distraction in orthopedic surgery might well be considered one of the earliest 
successful forms of tissue engineering. Bone distraction is a therapeutic process invented in 
Russia in about 1951 for treating fractures, lengthening limbs and correcting other skeletal 
30 defects such as angular deformities. In bone distraction, external fixators are used to correct 
bone deformities and to lengthen bones by the controlled application of 'tension-stress', 
resulting in natural, healthy tissue. 

Fig. 1 illustrates a prior art Dizarov fixator 20 attached to a bone 22. The external 
35 Kizarov fixator 20 is constituted of a pair of rings 24 separated by adjustable struts 28. The 
rings 24 are mounted onto the bone 22 from outside of the patient's body through wires or 

1 
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half-pins 26 as illustrated in Fig. 1. The lengths of the struts 28 can be adjusted to control the 
relative positions and orientations of the rings 24. After the fixator 20 is mounted to the 
patient's bone 22, the bone 22 is cut by osteotome (i.e., surgical cutting of a bone) as part of 
the bone distraction process. Thereafter, the length of each individual strut 28 is adjusted 
5 according to a surgical plan. This length adjustment results in the changing of the relative 
position of the rings 24, which then forces the distracted (or "cut") bone ends to comply and 
produce new bone in-between. This is termed the principle of "tension-stress" as applied to 
bone distraction. 

10 The bone distraction rate is usually controlled at approximately 1mm (millimeter) per 

day. The new bone grows with the applied distraction and consolidates after the distraction is 
terminated. Thereafter, the fixator 20 can be safely removed from the bone 22 and, after 
recanalization, the new or "distracted" bone is almost indistinguishable from the old or pre- 
surgery bone. The bone 22 may be equipped with other units, such as hinges, to correct 

15 rotational deformities about one or a few fixed axes. Thus, controlled application of 

mechanical stress forces the regeneration of the bone and soft tissues to correct their own 
deformities. The whole process of deformity correction is known as "bone distraction." 

At present, the following nominal steps are performed during the bone distraction 
20 process: (1) Determine an appropriate frame size for the fixator (e.g., for the Ilizarov fixator 
20); (2) Measure (e.g., from X-rays) the deformity of bone fragments (or the anticipated 
fragments after surgically cutting the bone) and obtain six parameters that localize one 
fragment relative to the other; (3) Determine (or anticipate) how the fixator frame should be 
mounted on the limb; (4) Input the parameters and measurements to a computer program that 
25 generates the strut lengths as a function of time required to correct the deformity; (5) Mount 
the fixator frame onto the bone fragments; and (6) Adjust the strut lengths on a daily basis 
according to the schedule generated in step (4). 

The steps outlined in the preceding paragraph are currently executed with minimal 
30 computerized assistance. Typically, surgeons manually gather or determine the required data 
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(e.g., fixator frame size, bone dimensions, fixator frame mounting location and orientation, 
etc) and make their decisions based on hand-drawn two-dimensional sketches or using 
digitized drawings obtained by tracing X-ray images. For example, a computerized deformity 
analysis (CD A) and pre-operative planning system (hereafter "the CDA system") developed 

5 by Orthographies of Salt Lake City, Utah, USA, creates the boundary geometry of bones 

using X-ray images that are first digitized manually, i.e., by placing an X-ray image on a light 
table and then tracing the outline with a digitizing stylus, and then the digital data are fed into 
the CDA system. Thereafter, the CDA system assists the surgeon in measuring the degree of 
deformity and to make a surgical plan. The entire process, however, is based on two- 

1 0 dimensional drawings and there is no teaching of showing or utilizing three-dimensional 
bone deformity or bone geometry. 

It is observed that in the complex area of bone distraction surgery, it is difficult, if not 
impossible, to make accurate surgical plans based solely on a limited number of two- 

15 dimensional renderings of bone geometry. This is because of the complex and inherently 
three-dimensional nature of bone deformities as well as of fixator geometry. Furthermore, 
two-dimensional depictions of surgical plans may not accurately portray the complexities 
involved in accessing the target positions of the osteotome and fixator pins surrounding the 
operated bone. Lack of three-dimensional modeling of these geometric complexities makes it 

20 difficult to accurately mount the fixator on the patient according to the pre-surgical plan. 

After a surgeon collects the requisite data (e.g., fixator frame size to be used, patient's 
bone dimensions, fixator frame mounting location and orientation, etc.), the surgeon may use 
the simulation software accompanying commercially available fixators (such as the Taylor 

25 Spatial Frame distributed by Smith & Nephew Inc. of 1450 Brooks Road, Memphis, 

Tennessee, USA 381 16) to generate a day-by-day plan that shows how the lengths of the 
fixator struts should be adjusted. Such a plan is generated after the initial and target frame 
positions and orientations are specified by the surgeon. However, the only functionality of the 
simulation software is a simple calculation of the interpolated frame configurations. The 

30 software does not provide any assistance to the surgeon about making surgical plans nor does 
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it provide any visual feedback on how the fixator frame and bone fragments should be moved 
over time. 

The Taylor Spatial Frame (shown, for example, in Fig. 16) with six degrees of 
5 freedom (DOF) is more versatile, flexible and complex than the Hizarov fixator 20 in Fig. 1 . 
Because of the sophistication of modern fixators (e.g., the Taylor Spatial Frame) and because 
of the limitations of the presently available bone distraction planning and execution systems, 
current computerized bone distraction procedures are error-prone, even when performed by 
the most experienced surgeons. As a result, the patients must typically revisit the surgeon 

10 several times after the initial operation in order for the surgeon to re-plan and refine the 
tension-stress schedule, or even to re-position the fixator. Such reiterations of surgical 
procedures are not only time-consuming, but incur additional costs and may lead to poorer 
therapeutic results while unnecessarily subjecting patients to added distress. It is therefore 
desirable to generate requisite bone and fixator models in three-dimensions prior to surgery so 

15 as to minimize the surgery planning and execution errors mentioned hereinbefore. 

The discussion given hereinbelow describes some additional software packages that 
are available today to assist in the simulation and planning of bone distraction. However, it is 
noted at the outset that these software packages are not based on three-dimensional models. 

20 Further, these software packages are quite limited in their capabilities to assist the surgeon in 
making important clinical and procedural decisions, such as how to access the site of the 
osteotomy or how to optimally configure fixator pin configurations. Additional limitations of 
the present software systems include: (1) No realistic three-dimensional view of a bone and a 
fixator; (2) No usage of animation in surgical simulation; (3) Lack of an easy-to-use graphical 

25 user interface for user-friendliness; (4) No on-line database of standard or past similar cases 
and treatment data; and (5) No file input/output to store or retrieve previous case data. 

In "Correction of General Deformity With The Taylor Spatial Frame Fixator™" 
(1997), Charles J. Taylor refers to a software package from Smith & Nephew (Memphis, 
30 Tennessee) (hereafter "the Smith software") that utilizes the Taylor Spatial Frame for certain 

4 
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computations. However, the Smith software does not include any visual output to the user 
(i.e., the surgeon) and the user needs to enter all data via a dialog box. Being mechanical in 
nature, the strut locations in a fixator are static. However, the Smith software does not 
account for whether a strut can be set to all the lengths necessary during the bone correction 
5 process. Further, the Smith software cannot calculate corrections that are due to malrotation 
(of the fixator) only. 

As described hereinbefore, a software for computerized bone deformity analysis and 
pre-operative planning is developed by Orthographies of Salt Lake City, Utah, USA (hereafter 

10 "the Orthographies software"). The Orthographies software creates the boundary geometry of 
bones using X-ray images that are first digitized manually as previously mentioned. 
Thereafter, the Orthographies software assists the surgeon in measuring the degree of bone 
deformity and to make a surgical plan. The entire process, however, is based on two- 
dimensional drawings and there is no support for showing or utilizing three-dimensional bone 

15 deformity or bone geometry. However, it is difficult to make accurate surgical plans based on 
a few such two-dimensional renderings considering the complex, three-dimensional nature of 
bone deformities and fixator geometry, and also considering the complexity involved in 
accessing the target positions of the osteotomy and fixator pins. This inherently three- 
dimensional nature of bone geometry and fixator assembly also makes it difficult to 

20 accurately mount the fixator on the patient's bone according to the two-dimensional pre- 
surgical plan. For further reference, see D. Paley, H. F. Kovelman and J. E. Herzenberg, 
Ilizarov Technology, "Advances in Operative Orthopaedics," Volume 1, Mosby Year Book, 
Inc., 1993. 

25 The software developed by Texas Scottish Rite Hospital for Children utilizes 

primitive digitization of the radiographs to generate three-dimensional representations of 
bones without any simulation. Additionally, the generated models are very primitive and do 
not show any kind of detail on the bone. For further reference, see Hong Lin, John G. Birch, 
Mikhail L. Samchukov and Richard B. Ashman, "Computer Assisted Surgery Planning For 
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Lower Extremity Deformity Correction By The Ilizarov Method," Texas Scottish Rite 
Hospital for Children. 

The SERF (Simulation Environment of a Robotic Fixator) software has capability to 
5 represent a three-dimensional bone model. However, the graphical representations of the 

fixator frame and the bone by the SERF software are over-simplified. Furthermore, there is no 
mention of any user interface except for a dialog box that prompts a user (e.g., a surgeon) for 
a "maximum distance." Additional information may be obtained from M. Viceconti, A. 
Sudanese, A. Toni and A. Giunti, "A software simulation of tibial fracture reduction with 
1 0 external fixator," Laboratory for Biomaterials Technology, Istituto Rizzoli, Bologna, Italy, 
and Orthopaedic Clinic, University of Bologna, Italy, 1993. 

In "Computer-assisted preoperative planning (CAPP) in orthopaedic surgery," 
Orthopaedic Hospital, Medical College, University of Zagreb, Yugoslavia, 1990, Vilijam 
15 Zdravkovic and Ranko Bilic describe a CAPP and Computer Assisted Orthopedic Surgery 
system. The system receives feedback and derives a bone's geometry from two two- 
dimensional scans. However, this system still uses the less sophisticated and less complex 
Ilizarov fixator 20 (Fig. 1) instead of the more advanced Taylor Spatial Frame. 

20 In a computer-assisted surgery, the general goal is to allow the surgeon to accurately 

execute the pre-operative plan or schedule. One approach to fulfil this goal is to provide 
feedback to the surgeon on the relative positions and the orientations of bone fragments, 
fixator frame and osteotomy/coricotomy site as the surgical procedure progresses. These 
positions could be determined in real time by measuring, with the help of an infrared (IR) 

25 tracking system, the positions of infrared light emitting diode (LED) markers strategically 
placed on the fixator frame, on cutting tools and on the patient. The relative positions of all 
these objects (and deviations from the planned positions) could then be displayed via a 
computerized image simulation to give guidance to the surgeon operating on the patient. Such 
a feedback approach is currently used to help register acetabular implants in artificial hip 

30 surgery using an Optotrak optical tracking camera from Northern Digital Inc. of Ontario, 

6 
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Canada. The Optotrak camera is capable of tracking the positions of special LEDs or targets 
attached to bones, surgical tools and other pieces of operating room equipment. However, for 
use in a computer-aided bone distraction system, the Optotrak camera and additional display 
hardware are too expensive to consider for a widespread bone distraction commercialization 
5 strategy. 

It is estimated that, at present, less than 1% of orthopedic surgeons practice the bone 
distraction procedure and less than 5000 bone distraction cases are performed per year 
worldwide. Such relative lack of popularity may be attributed to the fact that learning the 
1 0 techniques for bone distraction is extremely demanding and time-consuming. Therefore, the 
average orthopedic surgeon does not perform these techniques. Thus, there is a significant 
number of patients for whom external fixation with distraction would be the treatment of 
choice, but because of the current complexity and cost limitations, these patients never benefit 
from advanced bone distraction procedures. 

15 

It is therefore desirable to develop a user-friendly (i.e., a surgeon-friendly) system that 
would make bone distraction a viable option for a much broader market of surgeons than are 
currently using this therapy. It is also desirable to devise a computer-based surgical planning 
service that simplifies frame fixation, decreases preoperative planning time and reduces the 

20 chances of complications, thereby making frame fixation a relatively physician-friendly 
technique. To facilitate acceptance of complex bone distraction procedures to a wider 
segment of orthopedic surgeons, it is further desirable to overcome two primary limitations 
present in current surgical planning and execution software: (1) the lack of three-dimensional 
visual aids and user-friendly simulation tools, and (2) the lack'of an accurate and economical 

25 registration (i.e., fixator mounting) scheme. 
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SUMMARY OF THE INVENTION 

The present invention contemplates a method of generating a computer-based 3D 
(three dimensional) model for a patient's anatomical part comprising defining a 3D template 
5 model for the patient's anatomical part; receiving a plurality of 2D (two dimensional) x- 
ray images of the patient's anatomical part; extracting 2D fiducial geometry of the patient's 
anatomical part from each of said plurality of 2D x-ray images; and deforming the 3D 
template model using the 2D fiducial geometry of the patient's anatomical part so as to 
minimize an error between contours of the patient's anatomical part and those of the 
10 deformed 3D template model. 

A computer assisted orthopedic surgery planner software according to the present 
invention may identify the 2D fiducial geometry of a patient's bone (or other anatomical part 
under consideration) on the 3D template bone model prior to deforming the 3D template bone 

15 model to substantially conform to the contours of the actual patient's bone. In one 

embodiment, after detecting the bone contour, the computer assisted orthopedic surgery 
planner software creates a 3D lattice in which the 3D template bone model is embedded. 
Thereafter, a free-form deformation process is applied to the 3D lattice to match with the 
contour of the patient's bone, deforming the 3D template bone model in the process. 

20 Sequential quadratic programming (SQP) techniques may be used to minimize error between 
2D X-ray images data and the deformed template bone data. 

In an alternative embodiment, a template polygonal mesh representing a standard 
parametric geometry and topology of a bone is defined. The template polygonal mesh is then 
25 converted into a deformable model consisting of a system of stretched springs and bent 
springs. Then, multiple X-ray images of the patient's bone are used to generate force 
constraints that deform and resize the deformable model until the projections of the deformed 
bone model conform to the input X-ray images. To further assist the bone geometry 
reconstruction problem, a standard library of image processing routines may be used to filter, 
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threshold and perform edge detection to extract two-dimensional bone boundaries from the 
X-ray images. 

In another embodiment, the present invention contemplates a computer-based method 
5 of generating a surgical plan comprising reading digital data associated with a 3D (three- 
dimensional) model of a patient's bone, wherein the digital data resides in a memory in a 
computer; and generating a surgical plan for the patient's bone based on an analysis of the 
digital data associated with the 3D model. A surgical planner/simulator module in the 
computer assisted orthopedic surgery planner software makes a detailed surgical plan using 

10 realistic 3D computer graphics and animation. The simulated surgical plan may be viewed on 
a display screen of a personal computer. The planner module may also generate a pre-surgery 
report documenting various aspects of the bone surgery including animation of the bone 
distraction process, type and size of fixator frame and its struts, a plan for mounting the 
fixator frame on the patient's bone, the location of the osteotomy/coricotomy site and the day- 

1 5 by-day length adjustment schedule for each fixator strut. 

In a still further embodiment, the present invention contemplates an arrangement 
wherein a computer assisted orthopedic surgery planner computer terminal is connected to a 
remote operation site via a communication network, e.g., the Internet. The computer assisted 

20 orthopedic surgery planner software may be executed on the computer assisted orthopedic 
surgery planner computer. A fee-based bone distraction planning (BDP) service may be 
offered via a network (e.g., the Internet) using the computer assisted orthopedic surgery 
planner software at the service provider's site. An expert surgeon at the service provider's site 
may receive a patient's X-ray data and other additional information from a remotely-located 

25 surgeon who will be actually operating on the patient. The remotely-located surgeon may be a 
subscriber to the network-based BDP service. The expert surgeon may analyze the X-ray data 
and other patient-specific medical data supplied by the remotely-located surgeon with the 
help of the computer assisted orthopedic surgery planner software executed on the computer 
assisted orthopedic surgery planner computer. Thereafter, the expert surgeon may send to the 

30 remotely-located surgeon over the Internet the 3D bone model of the patient's bone, a 
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simulated surgery plan as well as a complete bone distraction schedule generated with' the 
help of the computer assisted orthopedic surgery planner software of the present invention. 

The computer assisted orthopedic surgery planner software of the present invention 
5 makes accurate surgical plans based solely on a number of two-dimensional renderings of the 
patient's bone geometry. The software takes into account the complex and inherently three- 
dimensional nature of bone deformities as well as of fixator geometry. Furthermore, three- 
dimensional simulation of the suggested surgical plan realistically portrays the complexities 
involved in accessing the target positions of the osteotome and fixator pins surrounding the 
10 operated bone, allowing the surgeon to accurately mount the fixator on the patient according 
to the pre-surgical plan. 

With the computer-aided pre-operative planning and frame application and adjustment 
methods of the present invention, the duration of fixation (of a fixator frame) maybe reduced 

15 by an average of four to six weeks. Additionally, by lowering the frequency of prolonged 
fixations, substantial cost savings per patient may be achieved. Shortening of the treatment 
time and reduction of complications may lead to better surgical results and higher patient 
satisfaction. The use of the computer assisted orthopedic surgery planner software of the 
present invention (e.g., in an Internet-based bone distraction surgery planning service) may 

20 make the frame fixation and bone distraction processes physician-friendly by simplifying 
fixation, decreasing preoperative planning time, and reducing the chances of complications 
through realistic 3D simulations and bone models. Thus more surgeons may practice bone 
distraction, resulting in benefits to more patients in need of bone distraction. 
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BRIEF DESCRIPTION OF DRAWINGS 

Further advantages of the present invention may be better understood by referring to 
the following description taken in conjunction with the accompanying drawings, in which: 

5 

Fig. 1 illustrates a prior art Hizarov fixator attached to a bone; 

Fig. 2 depicts an exemplary setup to perform computer assisted orthopedic surgery 
according to the present invention; 

10 

Fig. 3 shows an exemplary operational block diagram for the three modules 
constituting the computer assisted orthopedic surgery planner software according to the 
present invention; 

15 Fig. 4 graphically illustrates exemplary computer screen displays generated upon 

execution of the computer assisted orthopedic surgery planner software of the present 
invention; 

Fig. 5 is an exemplary flowchart depicting operational steps performed by the 3D 
20 geometry reconstructor module of the computer assisted orthopedic surgery planner software; 

Fig. 6 shows front and side X-ray images of a bone and corresponding bone 
boundaries extracted therefrom; 

25 Fig. 7 portrays intersection of swept bone boundaries shown in Fig. 6; 

Fig. 8 displays an undeformed 3D template bone model with the patient's bone 
geometry reconstructed thereon; 
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Fig. 9 A shows free-form deformation parameters and lattices deformed according to 
the contour of the patient's bone; 

Fig. 9B illustrates a binary tree subdivision process on a control block; 

5 

Fig. 10 illustrates a template triangular mesh in a physical-based approach to bone 
geometry reconstruction; 

Fig. 11 illustrates extension springs and torsion springs defined over a deformable 
1 0 triangular mesh model; 

Fig. 12 depicts the deformed 3D geometric model and the deformed lattice for the 
patient's bone; 

1 5 Fig. 1 3 A depicts the initial error between an X-ray image and a deformed template 

bone generated using a three-cell lattice; 

Fig. 13B depicts the initial error between an X-ray image and a deformed template 
bone generated using an eight-cell lattice; 

20 

Fig. 14A depicts the final error between the X-ray image and the deformed template 
bone shown in Fig. 13 A; 

Fig. 14B depicts the final error between the X-ray image and the deformed template 
25 bone shown in Fig. 13B; 

Fig. 1 5 is an exemplary flowchart depicting operational steps performed by the 
surgical planner/simulator module of the computer assisted orthopedic surgery planner 
software according to the present invention; 

30 
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Fig. 16 is an exemplary three-dimensional surgical simulation on a computer screen 
depicting a fixator, a bone model and the coordinate axes used to identify the bone's 
deformity and the osteotomy site; 

Fig. 17 shows an example of a graphical user interface screen that allows a user to 
manipulate the 3D simulation shown in Figs. 4 and 16; 

Fig. 1 8 depicts post-surgery X-ray images of a patient's bone along with the X-ray 
image of the fixator mounted thereon; and 

Fig. 19 illustrates an exemplary fixator ring incorporating easily identifiable and 
detachable visual targets. 
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DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

Fig. 2 depicts an exemplary setup to perform computer assisted orthopedic surgery 
according to the present invention. A computer assisted orthopedic surgery planner computer 
5 30 is accessible to a surgeon in a remote operation site 32 via a communication network 34. 
In one embodiment, the communication network 34 maybe an ethernet LAN (local area 
network) connecting all the computers within an operating facility, e.g., a hospital. In that 
case, the surgeon and the computer assisted orthopedic surgery terminal 30 may be physically 
located in the same site, e.g., the operating site 32. In alternative embodiments, the 
1 0 communication network 34 may include, independently or in combination, any of the present 
or future wireline or wireless data communication networks, e.g., the Internet, the PSTN 
(public switched telephone network), a cellular telephone network, a WAN (wide area 
network), a satellite-based communication link, a MAN (metropolitan area network) etc. 

15 The computer assisted orthopedic surgery planner computer 30 may be, e.g., a 

personal computer (PC) or may be a graphics workstation. Similarly, the doctor at the remote 
site 32 may have access to a computer terminal (not shown) to view and manipulate three- 
dimensional (3D) bone and fixator models transmitted by the computer assisted orthopedic 
surgery planner computer 30. In one embodiment, the computer assisted orthopedic surgery 

20 planner terminal 30 may function as the surgeon's computer when the operating site includes 
the computer assisted orthopedic surgery planner computer 30. Each computer — the computer 
assisted orthopedic surgery planner computer 30 and the remote computer (not shown) at the 
operating site — may include requisite data storage capability in the form of one or more 
volatile and non- volatile memory modules. The memory modules may include RAM (random 

25 access memory), ROM (read only memory) and HDD (hard disk drive) storage. Memory 

storage is desirable in view of sophisticated computer simulation and graphics performed by 
the computer assisted orthopedic surgery planner software according to the present invention. 

The computer assisted orthopedic surgery planner software may be initially stored on 
30 a portable data storage medium, e.g., a floppy diskette 38, a compact disc 36, a data cartridge 
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(not shown) or any other magnetic or optical data storage. The computer assisted orthopedic 
surgery planner computer 30 may include appropriate disk drives to receive the portable data 
storage medium and to read the program code stored thereon, thereby facilitating execution of 
the computer assisted orthopedic surgery planner software. The computer assisted orthopedic 

5 surgery planner software, upon execution by the computer assisted orthopedic surgery planner 
computer 30, may cause the computer assisted orthopedic surgery computer 30 to perform a 
variety of data processing and display tasks including, for example, display of a 3D bone 
model of the patients bone on the computer screen 40, rotation (on the screen 40) of the 3D 
bone model in response to the commands received from the user (i.e., the surgeon), 

10 transmitting the generated 3D bone model to the computer at the remote site 32, etc. 



Before discussing how the computer assisted orthopedic surgery planner software 
generates 3D bone and fixator models and simulates surgical plans for bone distraction, it is 
noted that the arrangement depicted in Fig. 2 may be used to provide a commercial, network- 

15 based bone distraction planning (BDP) service. The network may be any communication 
network 34, e.g., the Internet. In one embodiment, the surgeon performing the bone 
distraction at the remote site 32 may log into the BDP service provider's website and then 
send X-ray images, photographs and/or video of the patient's bone along with pertinent 
patient history to an expert surgeon located at and operating the computer assisted orthopedic 

20 surgery computer 30. The expert surgeon may then assess the case to determine if distraction 
is a viable option and, if so, then use the computer assisted orthopedic surgery planner 
software residing on the computer assisted orthopedic surgery computer terminal 30 to help 
plan the distraction process. The expert surgeon may transmit the distraction plan, simulation 
videos and distraction schedule — all generated with the help of the computer assisted 

25 orthopedic surgery planner software according to the present invention — to the service user 
(i.e., the surgeon at the remote site 32). Such a network-based bone distraction planning and 
consultancy service may be offered to individual surgeons or hospitals on a fixed-fee basis, on 
a per-operation basis or on any other payment plan mutually convenient to the service 
provider and the service recipient. 

30 
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In an alternative embodiment, the network-based bone distraction planning service 
may be implemented without the aid of the computer assisted orthopedic surgery planner 
software of the present invention. Instead, the expert surgeon at the computer assisted 
orthopedic surgery planner terminal 30 may utilize any other software or manual assistance 
5 (e.g., from a colleague) to efficiently evaluate the bone distraction case at hand and to 
transmit the response back to the surgeon or user at the remote site 32. 

Fig. 3 shows an exemplary operational block diagram for the three modules 
constituting the computer assisted orthopedic surgery planner software according to the 

10 present invention. The three modules are denoted by circled letters A, B and C. Module A is a 
3D geometry reconstructor module 42 that can generate a 3D bone geometry (as shown by the 
data block 44) from 2D (two-dimensional) X-ray images of the patient's bone as discussed 
hereinbelow. Module B is a surgical planner/simulator module 46 that can prepare a surgical 
plan for bone distraction (as shown by the data block 48). Finally, module C is a database 

15 module 50 that contains a variety of databases including, for example, a 3D template 

geometry database 52, a deformation mode database 54, a fixator database 56, a surgical tool 
database 58 and a surgical plan database 60. All of these modules are shown residing (in a 
suitable memory or storage area) in the computer assisted orthopedic surgery planner terminal 
30. The discussion hereinbelow focuses on modules A, B and C; however, it is understood 

20 that these modules do not function independently of a platform (here, the computer assisted 
orthopedic surgery planner computer 30) that executes the program code or instructions for 
the respective module. In other words, the screen displays and printouts discussed 
hereinbelow may be generated only after the program code for a corresponding module is 
executed by the computer assisted orthopedic surgery planner computer 30. 

25 

The 3D geometry reconstructor module (or module A) 42 according to the present 
invention reconstructs three-dimensional bone geometry using free-form deformation (FFD) 
and sequential quadratic programming (SQP) techniques. Module A also generates relative 
positions and orientations of the patient's bone and the fixator mounted thereon. The surgical 
30 planner/simulator moduel (or module B) 46 provides a user-friendly simulation and planning 
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environment using 3D, interactive computer graphics. Module B can show a realistic image 
of the bones, fixator and osteotomy/coricotomy, while the bone lengthening and deformity 
correction process is animated with 3D graphics. The database module (or module C) 50 aids 
in the measurement of the relative positions of the mounted fixator, osteotomy/coricotomy, 
5 and bones and feeds this information back into the computer assisted orthopedic surgery 
planner software to determine the final daily distraction schedule. 

As an overview, it is noted that the 3D geometry reconstructor module 42 takes two 
(or more than two) X-ray images of patient's bone, wherein the X-ray images are taken from 

10 two orthogonal directions.Module A 42 starts with a predefined three-dimensional template 
bone shape, whose shape is clinically normal and is scaled to an average size. Module A 42 
then scales and deforms the template shape until the deformed shape gives an image similar 
to an input X-ray image when projected onto a two-dimensional plane. Hierarchical free-form 
deformation (FFD) may be used to scale and deform the template bone, wherein the 

1 5 deformation in each deformation layer may be controlled by a number of variables (e.g., eight 
variables). Thus, the problem of finding the three-dimensional shape of the bone is reduced to 
an optimization problem with eight design variables. Therefore, one objective of module A 
42 is to minimize the error, or the difference, between the input X-ray image and the 
projected image of the deformed template shape. SQP (sequential quadratic programming) 

20 techniques may be used to solve this multi-dimensional optimization problem. In other words, 
SQP techniques may be applied to calculate optimized FFD parameters for least error. 

Generation of a 3D model of a patient's bone (or any other anatomical part) based on 
two or more X-ray images of the bone allows for efficient pre-, intra-, and post-operative 

25 surgical planning. It is noted that X-ray image-based shape reconstruction (e.g., generation of 
3D models of an anatomical part) is more computationally efficient, cost effective and 
portable as compared to image processing using standard three-dimensional sensor-based 
methods, such as MRI (magnetic resonance imaging) or CAT (computerized axial 
tomography).. The three-dimensional shapes generated by Module A 42 may be useful in 

30 many applications including, for example, making a three-dimensional physical mockup for 
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surgery training or importing into and using in a computer-aided planning system for 
orthopedic surgery including bone distraction and open/closed wedge osteotomy. 
Furthermore, module A may reconstruct the 3D geometric model of the bone even if there are 
partially hidden bone boundaries on X-ray images. 

5 

Using CAT or MRI data for reconstructing bone geometry, however, has several 
practical limitations. First, compared to X-ray images, CAT and MRI are not cost or time 
effective, which may inhibit widespread clinical usage. X-ray imaging is available not only in 
large medical institutes, but also in smaller medical facilities that cannot afford CAT or MRI 

10 equipment. Second, X-ray imaging is portable so that it can be used in a remote site, even in a 
battlefield. In addition, the cost of scanning each patient using CAT or MRI is high, and the 
procedure is time consuming. Another disadvantage of using MRI or CAT is associated with 
the robustness of the software that performs surface geometry extraction. CAT or MRI's 
volumetric data has a much lower resolution compared to X-ray images, and the surface 

15 extraction process often cannot be completed due to the low resolution. Finally, X-ray 
imaging is preferred for imaging osseous tissues. 



Because there is an unknown spatial relationship between the pre-operative data (e.g., 
medical or X-ray images, surgical plans, etc.) and the physical patient on the operating room 

20 table, the 3D geometry reconstructor module 42 provides for both pre-operative and intra- 
operative registration of orthopedic bone deformity correction. A 3D solid model of the bone 
generated by module A 42 (as shown by data block 44 in Fig. 3 and 3D bone image 67 in Fig. 
4) may function as a fundamental tool for pre-, intra-, and post-operative surgical planning. 
The 3D geometry reconstructor module 42 develops interactive, patient-specific pre-operative 

25 3D bone geometry to optimize performance of surgery and the subsequent biologic response. 



Fig. 4 graphically illustrates exemplary computer screen displays generated upon 
execution of the computer assisted orthopedic surgery planner software of the present 
invention. Figures 3 and 4 may be viewed together to better understand the functions 
30 performed by modules A, B and C, and also to have a visual reference of various 3D models 
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generated by the computer assisted orthopedic surgery planner software according to the 
present invention. Furthermore, Fig. 5 is an exemplary flowchart depicting operational steps 
performed by the 3D geometry reconstructor module 42 of the computer assisted orthopedic 
surgery planner software. The following discussion will also refer to various operational steps 
5 in Fig. 5 as appropriate. 

Initially, at block 62, a surgeon determines (at a remote site 32) which of the patient's 
anatomical parts (e.g., a bone) is to be operated on. Fig. 4 shows a bone 63 that is to be 
distracted. Thereafter, at block 64, the surgeon or an assistant of the surgeon prepares 

10 digitized X-ray images for various X-ray views of the patient's bone 63. Digitization may be 
carried out manually, e.g., by placing an X-ray image on a light table and then tracing the 
outline of the bone contour with a digitizing stylus. In the embodiment illustrated in Fig. 4, 
digitized versions of a lateral (Lat) X-ray image 65 and an anterior/posterior (AP) X-ray 
image 66 of the bone 63 are input to the computer assisted orthopedic surgery planner 

15 software via the communication network 34 interconnecting the remote patient site 32 and the 
computer assisted orthopedic surgery planner terminal 30. It is noted that the X-ray images 
• 65,66 represent bone geometry in two-dimensional (2D) views. 

Upon execution of module A (at step 82 in Fig. 5), module A 42 receives (at block 84 
20 in Fig. 5) as input the digitized X-ray images 65,66. It is assumed that the X-ray images 65,66 
are taken from two orthogonal directions, usually front (or AP) and side (or lateral). This 
constraint of the orthogonal camera positions is a strong one, but it may be loosened, if 
necessary, with the modification of deformation parameters and extra computational cost in 
the optimization process. Module A 42 may also receive positional data for the X-ray camera 
25 (not shown) with reference to a pre-determined coordinate system. Such coordinate position 
may be useful for module A 42'to "read" the received X-ray images 65,66 in proper 
geometrical context. A user, e.g., the operator of the X-ray camera, may manually input the 
camera position coordinates and viewing angle data. Alternatively, a scheme may be devised 
to automatically incorporate the camera position parameters and viewing angle data as a set of 
30 variables to be optimized during the optimization process discussed hereinbelow. More than 
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two X-ray images could be added to the input if greater accuracy is required or if a certain 
part of the bone that is hidden in the AP and lateral views plays an important role in the bone 
distraction procedure. Since MRI and CAT have volumetric data set, using X-ray images to 
reconstruct the bone structure (e.g., the 3D geometric module 69) is more cost-effective and 
5 less time-consuming. 

After receiving the 2D X-ray images 65,66, the 3D geometry reconstructor module 42 
may extract at step 86 the fiducial geometry (or bone contour) from the X-ray images. The 2D 
X-ray images 65,66 represent the bone contour with a set of characteristic vertices and edges 

10 with respect to the respective X-ray image's coordinate system. In one embodiment, an 
operator at the computer assisted orthopedic surgery planner terminal 30 may manually 
choose (with the help of a keyboard and a pointing device, e.g., a computer mouse) the bone 
contour from the 2D X-ray images 65,66 of the bone 63 displayed on the computer screen 40. 
In another embodiment, commercially available edge detection software maybe used to semi- 

1 5 automate the fiducial geometry extraction process. 

After, before or simultaneous with the fiducial geometry extraction, module A 42 may 
access the 3D template geometry database 52 to select a 3D template bone model (not shown) 
that may later be deformed with the help of the 2D X-ray images 65,66 of the patient's bone 

20 63. The size (or outer limits) of the 3D template bone model may be selected based on the 

computation of the closed volume that tightly bounds the patient's bone geometry. Figs. 6 and 
7 illustrate certain of the steps involved in that computation. Fig. 6 shows front (66) and side 
(65) X-ray images of a bone and corresponding bone boundaries (108 and 110 respectively) 
extracted therefrom. Fig. 7 portrays the intersection of swept bone boundaries 108,110 shown 

25 in Fig. 6. The intersection of the bone boundaries defines a closed volume that may tightly 
bound the 3D template bone model and that closely resembles the volumetric dimensions of 
the patient's bone. 

After detecting the bone contour at step 86, module A 42 first identifies (at step 88) 
30 the corresponding fiducial geometry on the 3D template bone model prior to any deformation 
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discussed hereinbelow. Module A 42 also optimizes (at steps 90 and 92) the 3D positioning 
and scaling parameters for the 3D template bone model until the size and position of the 3D 
template bone model is optimum with respect to the patient's bone 63 (as judged from the X- 
ray images 65,66 of the patient's bone 63). Upon finding the optimum values for positioning 
5 and scaling parameters, module A 42 updates (at step 94) the 3D template bone model with 
new positioning and scaling parameters. The resultant 3D template bone model 1 12 is shown 
in Fig. 8, which displays the undeformed 3D template bone model 112 with the patient's bone 
geometry reconstructed thereon. Module A 42 may also update (block 93) the 3D template 
geometry database 52 with the optimum positioning and scaling parameter values computed 
10 at steps 90 and 92 for the selected template bone model. Thus, the 3D template geometry 
database 52 may contain 3D template bone models that closely resemble actual, real-life 
patients' bones. 

In one embodiment, the 3D geometry reconstructor module 42 creates a 3D lattice 114 
15 in which the template bone 112 from Fig. 8 is embedded. A free-form deformation process is 
applied to this 3D lattice 1 14 in order to optimally match with the contour of the patient's 
bone. For the sake of simplicity, a few of the free-form deformation (FFD) parameters are 
shown in Fig. 9A and identified as a^ b i? and r. (where i=l to 4) in the x-y-z coordinate 

system for each parallelpiped (118, 120 and 122) in the 3D lattice 1 14. It may be desirable to 
20 have the 3D lattice 1 14 watertight in the sense that there may not be any gap and overlap 

between the faces of each constituent parallelpiped (118, 120 and 122) so as not to adversely 
affect a physical mockup made with a rapid prototyping process. In one embodiment, 
Sederberg and Parry's technique (hereafter "Parry's technique") may be used to reconstruct 
three-dimensional geometric model of the patient's bone. A detailed description of Parry's 
25 technique may be found in T. W. Sederberg and S. R. Parry, "Free Form Deformation of 
Solid Geometric Models," presented at SIGGRAPH '86 Proceedings, Dallas, Texas (1986), 
which is incorporated herein by reference in its entirety. 

It is stated in A. H. Barr (hereafter "Barr"), "Global and Local Deformations of Solid 
30 Primitives," Computer Graphics, vol. 18, pp. 21-30 (1984), which is incorporated herein by 
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reference in its entirety, that 'Deformations allow the user to treat a solid as if it were 
constructed from a special type of topological putty or clay which may be bent, twisted, 
tapered, compressed, expanded, and otherwise transformed repeatedly into a final shape." 
Barr uses a set of hierarchical transformations for deforming an object. This technique 
5 includes stretching, bending, twisting, and taper operators. However, Parry's technique 
deforms the space (e.g., the parallelpiped 3D lattice 1 14 in Fig. 9A) in which the object is 
embedded (as shown in Fig. 12). On the other hand, Coquillart 's Extended Free-Form 
Deformation (EFFD) technique changes the shape of an existing surface either by bending the 
surface along an arbitrarily shaped curve or by adding randomly shaped bumps to the surface 
10 using non-parallelpiped type 3D lattices as discussed in S. Coquillart, "Extended Free-Form 
Deformation: A Sculpturing Tool for 3D Geometric Modeling," Computer Graphics, vol. 24, 
pp. 187-196 (1990) and in S. Coquillart, Extended Free-Form Deformation: A Sculpturing 
Tool for 3D Geometric Modeling," INRIA, Recherche, France 1250 (June 1990), both of 
these documents are incorporated herein by reference in their entireties. 

15 

Here, Parry's FFD technique is applied to a new area of application, i.e., three- 
dimensional shape reconstruction from two-dimensional images, instead of to the traditional 
application domains of geometric modeling and animation. Additionally, hierarchical and 
recursive refinement is applied to the control grid of FFD to adjust the deformation 
20 resolution. Hierarchical refinement may be necessary because of the unique nature of the 

shape reconstruction problem, i.e., lack of a priori knowledge of the complexity or severity of 
the deformation. 

The basic idea of Parry* s technique is that instead of deforming the object (here, the 
25 3D template bone) directly, the object is embedded in a rectangular space that is deformed (as 
illustrated by Fig. 12). One physical and intuitive analogy of FFD is that a flexible object may 
be visualized as being "molded" in a clear plastic block and the whole block is deformed by 
stretching, twisting, squeezing, etc. As the plastic block is deformed, the object trapped inside 
the block is also deformed accordingly. Parry's technique uses the following single Bezier 
30 hyperpatch to perform this deformation: 
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q(u,v,w) = i%JptBMBj(»)B A (w),0 <u<*1,0<vZ1,0Zw<>1 ....(1) 

i=0 j=0 k-0 

where u, v, and w are parameter values that specify the location of an original point in the 
control block space, q(w, v, w) specifies the location of the point after the deformation, 
specifies points that define a control lattice, and B z {w), B y (v), and B^(w) are the Bernstein 
polynomials of degree n, for example: 

B i (u) = -^u i (l-u) n - i ....(2) 
i!(n-i)! 

In equation (2), a linear version of FFD as a unit deformation block (i.e., n=l) may be used. 
This is the simplest deformation function, and there are only eight control points used to 
define a control block for deformation — these eight points define eight corner points of a 
deformation block (e.g., as shown by the corner points of each parallelpiped in the 3D lattice 
1 14 in Fig. 9 A). The variation of a deformation with a linear function is limited compared to 
a higher order function, but a linear function may be preferable because the complexity of the 
deformation of a bone is unknown a priori. It may also be desirable to increase the resolution 
of a deformation as needed by using adaptive refinement of the control block. 

The adaptive refinement may be performed by using a hierarchical, recursive binary 
tree subdivision of the control block 123 as shown in Fig. 9B. A binary tree subdivision may 
be preferable rather than a more standard spatial subdivision of octree subdivision, because of 
the cylindrical or rim-type shape of the target bones (i.e., bones to be operated on) of a human 
patient. Octree may be a better choice when the target bone shape is not cylindrical. 
Furthermore, the extension from a binary subdivision to an octree subdivision may be 
straightforward. 

Parry's technique calculates the deformed position X ffd of an arbitrary point X, which 

has {s, t, u) coordinates in the system given by the following equation: 

X = X 0 + sS + fT + uU ....(3) 

The {s, t, u) coordinates are computed from the following equations: 
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10 



j = 



TxU.(X-X 0 ) 
TxU.S 



SxU.(X-X 0 ) 
SxU.T 

SxT.(X-X.) 
SxT.U 



-..(4) 
....(5) 
....(6) 



A grid of the control points, in equation (7) is imposed on each parallelpiped (1 18, 120 

and 122). This forms /+1 planes in the S direction, m+1 planes in the T direction, and n+1 
planes in the U direction. 

P iik = X o + T S+ - T+ - U -0) 
The deformation is then specified by moving the from their undisplaced, lattical positions 
according to the following equation: 



f-0 \ l J 



.(8) 



A sequential quadratic programming (SQP) algorithm may then be used to compute 
free form deformation (FFD) parameters (a^ b i and ^ in Fig. 9A) that minimize the error 

between the X-ray image and the deformed bone image. Because the 3D geometry 
15 reconstructor module 42 creates three connected parallelpipeds (1 1 8, 120 and 122 in Fig. 9 A), 
there are a total of eight parameters subject to optimization. More accuracy (i.e., minimization 
of error) may be achieved with increasing the number of parallelpiped lattices and also by 
increasing the number of FFD parameters. Before calculating the error, module A 42 may 
shrink the template bone data and the X-ray image data into a unit cube for convenient 
20 computation. The objective function of this minimization problem can be defined as follows: 

where Y n represents points on the boundary of an X-ray image; Q 7I represents points on the 
deformed bone template; and a v a 2 , etc. represent all deformation parameters (i.e., a i , b A and 
rj in Fig. 9A). If there is no error between the X-ray image under consideration and the 
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deformed bone image, and if the X-ray image is perfectly oriented, then the objective function 
in equation (9) above becomes zero. 

Steps 95-102 in Fig. 5 depict the process of optmizing the FFD parameters and, hence, 
5 minimizing the error (in equation (9)) between a corresponding 2D X-ray image (e.g., the 
lateral view 65 or the AP view 66 or any other available view) and the appropriate view of the 
3D template bone geometry 112 projected onto that X-ray image. Module A 42 projects (at 
step 95) the appropriate view of the 3D template bone geometry 112 onto the corresponding 
2D X-ray image (e.g., views 65 or 66 in Fig. 4) and calculates the matching error (at step 96) 
1 0 between the projection and the X-ray image. Based on the error calculation, module A 42 
attempts to optimize the FFD parameters at steps 98 and 100. The optimized values for the 
FFD parameters may then be used to generate the deformed polygonal mesh 1 16. At step 102, 
the 3D template bone model 1 12 is updated (i.e., deformed) with the new deformed 
polygonal mesh 116 talcing into account the new deformation parameters. 

15 

The process outlined by steps 84-102 is continued for each new X-ray image (e.g., for 
the lateral view 65 as well as for the AP view 66 in Fig. 4) as indicated by the decision block 
104. The process terminates at step 106 and the 3D geometry reconstructor module 42 outputs 
the final 3D bone geometry data (block 44 in Figs. 3 and 4) in the form of a 3D deformed 
20 bone model 69 for the patient's bone 63. The optimized values of FFD parameters obtained 
for a specific 3D template bone corresponding to a given bone contour (e.g., the patient's 
bone 63) may be stored in the deformation mode database 54 for future reference as well as to 
facilitate 3D viewing. The 3D solid bone model 69 may then be viewed by the surgeon at the 
remote site 32 for further surgical planning as depicted by block 68 in Fig. 3. 

25 

Certain of the steps discussed hereinbefore with reference to Fig. 5 are depicted in 
Figs. 12, 13 and 14. Fig. 12 depicts the deformed 3D geometric model 69 and the deformed 
lattice 1 16 for the patient's bone 63. Fig. 13A depicts the initial error between an X-ray image 
132 and a deformed template bone 130 generated using a lattice with three cells or three 
30 parallelpipeds (e.g., the lattice 1 14 in Fig. 9A). Fig. 13B, on the other hand, depicts the initial 
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error between an X-ray image 132 and a deformed template bone 1 30 generated using a 
lattice with eight cells or eight parallelpipeds (e.g., the lattice resulting from the binary tree 
subdivision of the control block 123 in Fig. 9B). Due to significant errors in Figs. 13 A and 
13B, the optimization process at steps 98, 100 (Fig. 5) may continue to minimize the 
5 projection error (i.e., to continue deforming the template bone 130). Fig. 14A depicts the final 
error between the X-ray image 132 and the deformed template bone 130 shown in Fig. 13 A. 
In other words, Fig. 14A shows the final error in a deformation process that uses a lattice with 
three cells (e.g., the lattice 1 14 in Fig. 9A). On the other hand, Fig. 14B depicts the final error 
between the X-ray image 132 and the deformed template bone 130 shown in Fig. 13B. In 

10 other words, Fig. 14B shows the final error in a deformation process that uses a lattice with 
eight cells or eight parallelpipeds (e.g., the lattice resulting from the binary tree subdivision of 
the control block 123 in Fig. 9B).The eventually deformed template bone 134 may have bone 
geometry that closely resembles that of the patient's bone 63. The entire 3D bone model 
generation process depicted in Fig. 5 may be implemented in any suitable programming 

15 language, such as, e.g., the C++ programming language, and may be executed on any suitable 
computer system, such as, e.g., a personal computer (PC), including the computer assisted 
orthopedic surgery planner computer 30. The final deformed bone geometry 69 may be 
displayed on the display screen 40 (Fig. 2) and may also be sent to the surgeon at the remote 
site 32 over the communication network 34 as discussed hereinbefore. 

20 

In an alternative embodiment, a physical-based approach may be used to create a 3D 
solid (or deformed) template bone model (i.e., the model 69 in Fig. 4) that may later be used 
by the surgeon at the remote site 32 for, e.g., mockup surgery practice. As part of the 
deformation process, first, a template polygonal mesh that represents a standard parametric 
25 geometry and topology of a bone is defined. The length and girth of the polygonal mesh is 
scaled for each patient based on the size of the corresponding 3D template bone model (e.g., 
the 3D template bone model 1 12 in Fig. 8). A model consisting of parametric surfaces, such 
as B6zier surfaces and non-uniform rational B-spline (NURBS) surfaces may provide 
increased resolution. Fig. 10 illustrates a template triangular mesh 124 in a physical-based 
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approach to bone geometry reconstruction. The contours of the 3D template bone model 112 
(Fig. 8) may be visualized as being composed of the triangular mesh 124. 

Thereafter, the template polygonal mesh (here, the triangular mesh 124) is converted 
5 into a deformable model consisting of a system of stretched springs and bent springs. Fig. 1 1 
illustrates extension springs (e^ and torsion springs (t^ defined over a deformable triangular 

mesh model 125. Then, multiple X-ray images (e.g., images 65 and 66 in Fig. 4) are used to 
generate force constraints that deform and resize the deformable model 125 until the 
projections of the deformed bone model conform to the input X-ray images as shown and 
10 discussed hereinbefore with reference to Figs. 13 and 14. A standard library of image 

processing software routines that filter, threshold and perform edge detection may be used to 
extract (for comparison with the projections of the deformed bone model) the two 
dimensional bone boundaries from the X-ray images as discussed hereinbefore. 

15 Referring now to Fig. 1 1, the extension springs (e^ are defined over the edges 126 and 

the torsion springs (t.) are defined over the edges 128 for a node 129 under consideration. It is 

assumed that the original length of an extension spring is given by an edge (e.g., the edge 
126) of the template polygon mesh (here, the triangular mesh 125) so that the tensile force is 
proportional to the elongation of that edge. The spring constant of an extension spring may be 
20 denoted as fi k\lt is also assumed that the original angle of a torsion spring is given by the 
template mesh (here, the mesh 125) so that the torque exerted by the torsion spring is 
computed based on the angular displacement. The spring constant of a torsion spring may be 
denoted as *fif. 

25 The total force T exerted on a node (e.g., the center node 129) is calculated by 

summing: (1) the tensile forces c f ' applied by all the extension springs attached to the node, 

and (2) the forces 6 f tf 9 applied by all the torsion springs surrounding the node 129. In the 
deformable triangular mesh model 125, five extension springs e f (i = 1 to 5) and five torsion 
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10 



springs t z - (i = 1 to 5) exert forces on the center node 129. The total force T is thus calculated 
as the summation of the forces from all the springs as given by the following equation: 

N N on 

-(10) 

;-l M h 

where iVis the number of edges attached to the node (here, the center node 129). Thus, TV is 
equal to the number of triangles surrounding the node. Furthermore, in equation (10), d t is the 

length of the extension spring 0 i is the angle between the normal vectors of the two 

triangles that share the torsion spring ^ as a common edge, and lj is the perpendicular distance 

from the node (here, the center node 129) to the torsion spring tj. 

By defining the equation of motion of this spring system and by numerically 
integrating the equation of motion, an equilibrium configuration of the spring system that 
minimizes the potential energy of the system can be given by the following equation: 

u=ii±~^ + ±u 0 A ....(id 

all V i=l L i=l Z J 



15 Thus, each triangle in the deformable triangular mesh 125 may get deformed according to the 
force constraints generated by the resulting mismatch (at steps 95,96 in Fig. 5) when the 
image of the 3D template bone geometry 112 (Fig. 8) is projected onto a corresponding 2D X- 
ray image (e.g., the lateral view 65, the AP view 66, etc.). The deformation of the triangular 
mesh 125 may continue until the matching error is minimized as indicated by steps 96, 98,100 

20 and 102. Upon minimization of the matching error, an equilibrium condition may get 
established as given by equation (1 1). The equilibrium process outlined above for the 
triangular mesh spring model of Figs. 10 and 1 1 may be repeated for each X-ray image of the 
patient's bone 63 as denoted by the decision step 104 in Fig. 5. 

25 Fig. 15 is an exemplary flowchart depicting operational steps performed by the 

surgical planner/simulator module (or module B) 46 of the computer assisted orthopedic 
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surgery planner software according to the present invention. Module B 46 assists a surgeon in 
making a detailed surgical plan by utilizing accurate 3D bone models (generated by module A 
42) and realistic 3D computer graphics and animation. Upon initial execution (at step 136), 
the planner module 46 reads or takes as an input (at step 138) the 3D geometry of the 
5 patient's anatomical part (here, the patient's bone 63). This 3D geometry may have been 

generated earlier by the 3D geometry reconstructor module 42 as discussed hereinbefore with 
reference to Figs. 5-14. Thereafter, the surgeon viewing the 3D bone model 69 may determine 
(at step 140) whether any similar past case exists where the bone treated had similar 3D 
geometry as the current patient's bone 63. The surgeon may make the decision either upon 
10 manual review of the patient's 3D bone geometry 69 or using the surgical plan database 58 or 
any similar data storage. Alternatively, module B 46 may perform similar decision-making 
based on a comparison with the data stored in the surgical plan database 60. 

If there is a past case that involves a bone having similar 3D geometry as the current 
15 patient's bone 63, then the surgeon may instruct (at step 142) module B 46 to read the 

surgical data associated with the past case from the surgical plan database 60. Alternatively, 
upon finding a matching or similar past case, module B 46 may automatically perform a 
search of the surgical plan database 60 to retrieve and send pertinent past surgical data to the 
surgeon at the remote site 32 so that the surgeon may determine whether to follow the steps 
20 performed earlier in another case or to alter or improve the earlier executed surgical plan. 
Whether there is a past similar case or not, the surgical planner module 46 generates a 
specification of the osteotomy site(s) and of the target geometry (e.g., the mounting 
arrangement 75 in Fig. 4) at step 144. Thereafter, at step 146, the planner module 46 may 
access the fixator database 56 to select the appropriate fixator type (e.g., the Hizarov fixator 
25 20 of Fig. 1 or the Taylor Spatial Frame 162 of Fig. 16). Further, during step 146, the planner 
module 46 may also generate information about the least intrusive mounting location for the 
fixator selected. 

Module B (i.e., the planner module 46) may further continue the optimum and most 
30 efficient surgical plan generation process by selecting (at step 148), from the surgical tool 
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database 58, appropriate surgical tools that may be needed to perform osteotomy or bone 
distraction on the patient's bone 63. Module B 46 may take into account the 3D geometry of 
the template bone model 69 generated by module A 42 to determine the most useful set of 
tools for the desired surgical procedure. The surgical planner module 46 then performs an 
5 analysis (at step 150) of how easily accessible the osteotomy site (specified earlier at step 
144) is with the current selection of surgical tools (at step 148). The surgical planner module 
46 may analyze (at the decisional step 152) its accessibility determination at step 150 based 
on, for example, an earlier input by the surgeon as to the kind of surgery to be performed on 
the patient's bone 63 and also based on the contour data available from the 3D template bone 
10 geometry generated by module A 42. If the planner module 46 determines any difficulty (e.g., 
difficulty in mounting the fixator or difficulty in accessing the osteotomy site, etc.) with the 
currently determined accessibility approach, then the planner module 46 may reevaluate its 
earlier determinations as shown by the iteration performed at step 152. 

15 Upon determining a viable (i.e., easily accessible and least intrusive) surgical plan for 

the patient's bone 63, the planner module 46 may further prepare a time-line for the bone 
distraction operation (at step 156) based on a decision at step 154. The surgeon at the remote 
site 32 may specify prior to executing the computer assisted orthopedic surgery planner 
software whether bone distraction needs to be performed and whether the surgeon would like 

20 to have a computer-based time-line for the distraction process (including such steps as fixator 
mounting, daily adjustment of struts and final removal of the fixator). Finally, at step 158, the 
planner module 46 generates an optimum surgical plan 48 (Figs. 3 and 4) for the patient's 
bone 63 based on available bone geometry and other surgical data. Prior to ending at step 160, 
module B 46 may store the recommended surgical plan in the surgical plan database 60 for 

25 future reference (e.g., for case comparison in a future case) and may also send the plan 48 to 
the surgeon at the remote site 32 via the communication network 34. In one embodiment, the 
surgical plan 48 may include a report documenting: (1) animation of the bone distraction 
process, (2) type and size of the fixator frame and its struts, (3) a suggested fixator frame 
mounting plan, (4) the osteotomy/coricotomy site location, (5) locations of fixator pins, and 

30 (6) the day-by-day length adjustment schedule for each fixator strut. 
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The surgeon at the remote site 32 may view the suggested surgery plan 48 received 
from the computer assisted orthopedic surgery planner computer 30 as depicted by block 70 
in Fig. 3. The realistic 3D computer graphics and animation contained in the simulated 

5 surgery plan create a CAD (computer aided design) environment that can help a surgeon 

better understand the three-dimensional positional relationships between the bone, the fixator, 
the osteotomy/coricotomy site, and the fixator pins. Because the surgeon would be able to 
create and verify the operation plan using easy-to-understand three-dimensional views, a more 
precise plan could be made in a shorter period of time. In one embodiment, the three- 

10 dimensional graphics for the surgical plan 48 may be generated using the OpenGL (open 
graphics library) software interface developed by Silicon Graphics, Inc., of Mountainview, 
California, USA. The OpenGL graphics software interface may be implemented on a 
conventional PC (personal computer) platform to show animations of the bone distraction 
process. 

15 

The 3D simulation of the proposed surgical plan is depicted as the initial simulation 
72 in Fig. 4. The computer-assisted surgical simulation 72 depicts the 3D template bone 
geometry 69 for the patient's bone 63 with a Taylor Spatial Frame 73 mounted thereon 
according to the specifications computed by module B 46. The final location and orientation 
20 of the fixator frame 73 on the 3D solid bone model 69 is depicted by the simulated target 
position 75 in Fig. 4. Thus, the initial operational position 72 and the final or desired target 
position 75 are simulated by the surgical planner module 46 to guide the surgeon during the 
actual surgery. 

25 Fig. 16 also shows the initial three-dimensional surgical simulation 72 on a computer 

screen depicting the fixator 73, the 3D solid bone model 69 and the coordinate axes used to 
identify the bone's deformity and the osteotomy site. The location of the suggested cutting of 
the bone for the bone distraction is also visible in the 3D simulated model 72 in Fig. 16. 
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Fig. 17 shows an example of a graphical user interface (GUI) screen 162 that allows a 
user (e.g., a surgeon) to manipulate the 3D simulations 72 or 75 shown in Figs. 4 and 16. 
Thus, the surgeon at the remote site 32 may manipulate the 3D simulated models 72 or 75 

with a pointing device (e.g., a computer mouse) and through the Microsoft Windows® dialog 
5 box (or GUI) 162 appearing on the screen of the computer where the surgeon is viewing the 
3D models. Using the dialog box or the GUI 162 the surgeon may correct the stress-tension 
for the struts in the fixator frame 73 and view the simulated results prior to actually 
attempting the surgery. 

1 0 The surgeon may then perform the surgery as suggested by the surgical plan generated 

by the computer assisted orthopedic surgery planner software module B 46. X-ray imaging is 
again used to measure all the relative positions after the fixator frame (e.g., the Taylor Spatial 
Frame 73) has been actually mounted (at block 74 in Fig. 3) and after the 
osteotomy/coricotomy has been made by the surgeon. A computer-aided surgery module may 

1 5 measure the actual positions of the bone deformity relative to the attached fixator and 

coricotomy, and the surgeon at the remote site 32 may feedback or input the positional data 
generated by such measurement into the computer assisted orthopedic surgery planner 
software for final determination of the distraction schedule based on the actual surgical data. 
The feedback data from the actual surgery may be sent to the computer assisted orthopedic 

20 surgery planner computer 30 over the communication network 34 as shown by the post- 
surgery X-ray images data output from block 76 in Fig. 3. 

Fig. 18 depicts post-surgery X-ray images (164, 166) of a patient's bone along with 
the X-ray image (165) of the fixator mounted thereon. The X-ray image 164 may correspond 

25 to the post-surgery lateral view 78 and the X-ray image 166 may correspond to the post- 
surgery lateral view 80 shown in Fig. 4. The digitized versions of these post-surgery X-ray 
images 164, 166 may be sent to the computer assisted orthopedic surgery planner software as 
denoted by block 76 in Fig. 3. Upon receipt of the post-surgery X-ray data, the computer 
assisted orthopedic surgery planner software module B 46 may act on the data to identify 

30 deviation, if any, between the suggested surgical plan data and the actual surgery data. 

32 



WO 01/78015 



PCT/US01/11272 



Thereafter, module B 46 may revise the earlier specified distraction trajectory (at step 156 in 
Fig. 15) to assure a correct kinematic solution in view of any discrepancy between the pre- 
surgery plan data and the post-surgery data. Module B 46 may still optimize the distraction 
plan even if the fixator is not mounted exactly as pre-surgically planned. 

5 

In one embodiment, to facilitate imaging and measurement of the fixator's position, a 
modified design for the fixator ring may be used. Fig. 19 illustrates an exemplary fixator ring 
168 incorporating easily identifiable and detachable visual targets 170. The fixator ring 168 in 
Fig. 19 may be used as part of a ring for the Uizarov fixator 20 (Fig. 1) or the Taylor Spatial 

10 Frame 73 (Figs. 4 and 16). For example, the modified fixator ring 168 may replace the ring 24 
in the Ilizarov fixator 20 shown in Fig. 1. The geometrical feature or targets 170 may be 
easily identifiable in computerized X-ray images. In the embodiment shown in Fig. 19, three 
posts (or targets or markers) 170 are attached to the ring 168 with each post having a unique 
geometry (here, the number of groves on the post) to identify the marker's 170 position in the 

1 5 X-ray image of the corresponding fixator. More or less than three posts may also be utilized. 
Furthermore, one or more posts may include a target sphere 172 at their open ends as shown. 
Thus, the surgeon may easily identify the fixator as well as the orientation of the fixator on 
the patient's bone. 

20 After acquiring the X-ray image (e.g., a post-surgery X-ray image) and after 

performing automatic filtering, thresholding and edge detection on the X-ray image, the 
digitized X-ray image may be displayed on a window on a computer screen (e.g., the display 
screen 40 in Fig. 2 or a display screen of a computer at the remote site 32). The location of 
geometrical targets 170 may be done by a simple and reliable user-interactive mode. For 

25 example, the computer assisted orthopedic surgery planner computer 30 or the surgeon's 
computer at the remote site 32 may be configured to prompt the surgeon attending the 
computer to identify each target post 170 by moving the computer's cursor (or pointing with a 
computer mouse) over the approximate location of the marker's sphere 172 and then clicking 
to select. The computer may be configured (e.g., with a search software) to automatically 

30 search a bounded area to localize the sphere 1 72 and measure its relative position. This 
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process may be done in both the AP and the lateral views. Similarly, the 
osteotomy/coricotomy may be located by prompting the surgeon to draw a line with the 
cursor (or with a computer mouse) over the osteotomy's location in the X-ray images. 
Because the position of each sphere 172 relative to the ring 168 that it is attached to would be 
5 known a priori, the positions and orientations of all rings on a fixator frame could thus be 
measured relative to the osteotomy/coricotomy. The targets 170, 172 could be removed from 
the fixator rings 168 before discharging the patient. 

The foregoing describes exemplary embodiments of a computer assisted orthopedic 
10 surgery planner software according to the present invention. It is noted that although the 
discussion hereinabove focuses on the use of the computer assisted orthopedic surgery 
planner software for a patient's bone, the software may also be used for surgical planning and 
3D modeling of any other anatomical part of the patient's body. Some of the major areas of 
applications of the computer assisted orthopedic surgery planner software of the present 
15 invention include: (1) Bone deformity correction including (i) osteotomy planning, simulation 
and assistance for, e.g., long bone deformities, complex foot deformities, (ii) acute fracture 
stabilization and secondary alignment in multiple trauma, and (iii) distraction osteogenesis 
case planning, simulation and assistance for, e.g., congenital and acquired deformities; (2) 
Maxillofascial as well as plastic reconstructive surgery; (3) Telemedicine or web-based 
20 surgical planning for physicians at distant locations; (4) Aide in the design of custom 

prosthetic implants; (5) Axial realignment when doing cartilage joint resurfacing; and (6) 
Creation of anatomical models for education of students and surgeons (e.g., for mock practice 
of surgical techniques). 

25 The computer assisted orthopedic surgery planner software according to the present 

invention facilitates generation and simulation of accurate 3D models of a patient's 
anatomical part, e.g., a bone. Furthermore, in the complex area of bone distraction surgery, 
the computer assisted orthopedic surgery planner software makes accurate surgical plans 
based solely on a number of two-dimensional renderings or X-ray images of bone geometry. 

30 The software takes into account the complex and inherently three-dimensional nature of bone 
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deformities as well as of fixator geometry when preparing a simulation of the proposed 
surgical plan prior to actual surgery. Complexities involved in accessing the target positions 
of the osteotome and fixator pins surrounding the operated bone are substantially reduced 
with the help of CAD (computer aided design) tools and 3D simulation of surgical 
5 environment. Three-dimensional modeling allows for an accurate mounting of a fixator frame 
on the patient's bone according to a pre-surgical plan. 

An Internet-based bone distraction planning service maybe offered on a subscription- 
basis or on a per-surgery basis to surgeons located at remote places where computer assisted 
10 orthopedic surgery planner software may not be directly available. An expert surgeon may 
operate the service provider's computer assisted orthopedic surgery planner terminal to devise 
a surgical plan and distraction schedule for the remotely-located surgeon based on the X-ray 
image(s) data and other specific requests received from the remote surgeon over the Internet. 

15 As noted hereinbefore, there are fewer than 1% of orthopedic surgeons who practice 

bone distraction. Furthermore, the external fixation with distraction currently takes an average 
of twelve to sixteen weeks at a cost of $1800 per week. However, even more time is required 
if the fixator was not initially properly mounted as often occurs in complicated cases. In these 
cases, the distraction schedule must be changed or the fixator must be reinstalled. The risk of 

20 major complications, including bone infection or fixation to bone failure rises exponentially 
when treatment times are extended. Complications and reinstallation of the fixator can require 
additional surgery costing $5000 to $10,000 and further extending the duration of fixation. 

With the computer-aided pre-operative planning and frame application and adjustment 
25 methods described hereinabove, the duration of fixation (of a fixator frame) may be reduced 
by an average of four to six weeks. Additionally, by lowering the frequency of prolonged 
fixations, the cost savings may be approximately $9000 per patient. Shortening of the 
treatment time and reduction of complications may lead to better surgical results and higher 
patient satisfaction. The use of the computer assisted orthopedic surgery planner software of 
30 the present invention (e.g., in the Internet-based bone distraction surgery planning service) 
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may make the frame fixation and bone distraction processes physician-friendly by simplifying 
fixation, decreasing preoperative planning time, and reducing the chances of complications 
through realistic 3D simulations and bone models. 

While several embodiments of the invention have been described, it should be 
apparent, however, that various modifications, alterations and adaptations to those 
embodiments may occur to persons skilled in the art with the attainment of some or all of the 
advantages of the present invention. It is therefore intended to cover all such modifications, 
alterations and adaptations without departing from the scope and spirit of the present 
invention as defined by the appended claims. 
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CLAIMS 

What is claimed is: 

1 . A method of generating a computer-based 3D (three-dimensional) model for a patient's 
5 anatomical part comprising; 

defining a 3D template model for the patient's anatomical part; 
receiving a plurality of 2D (two-dimensional) X-ray images of the patient's anatomical 
part; 

extracting 2D fiducial geometry of the patient's anatomical part from each of said 
1 0 plurality of 2D X-ray images; and 

deforming the 3D template model using the 2D fiducial geometry of the patient's 
anatomical part so as to minimize an error between contours of the patient's 
anatomical part and those of the deformed 3D template model. 

15 2. The method of claim 1, wherein defining the 3D template model includes selecting the 3D 
template model from a set of 3D template models stored in an electronic database. 

3 . The method of claim 1 , wherein receiving the plurality of 2D X-ray images includes 
receiving said each of the plurality of 2D X-ray images in a digitized form. 

20 

4. The method of claim 1 , wherein receiving the plurality of 2D X-ray images includes: 
generating the plurality of 2D X-ray images of the patient's anatomical part; and 
digitizing said each of the plurality of 2D X-ray images. 

25 5. The method of claim 1, wherein the patient's anatomical part is a bone. 

6. The method of claim 1, wherein extracting 2D fiducial geometry includes detecting 
surface contours of the patient's anatomical part from said each of the plurality of 2D X- 
ray images. 

30 
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The method of claim 1, wherein deforming the 3D template model includes: 
identifying the 2D fiducial geometry of the patient's anatomical part on the 3D template 

model therefor; 
embedding the 3D template model into a 3D lattice; and 

deforming the 3D lattice until a projection error between one of the plurality of 2D X-ray 
images and a corresponding view of the 3D template model projected thereon is 
minimized. 

The method of claim 7, wherein deforming the 3D lattice includes: 
computing a plurality of free fomi defonnation (FFD) parameters for the 3D lattice; and 
optimizing values for the plurality of FFD parameters so as to minimize the error between 
the contours of the patient's anatomical part and those of the deformed 3D template 
model. 

15 9. The method of claim 7, wherein the 3D lattice is constituted of a plurality of 
parallelpipeds. 

10. The method of claim 1, wherein said each of said plurality of 2D X-ray images is 
contained in a plane that is orthogonal to the planes containing the remainder of said 

20 plurality of 2D X-ray images. 

1 1. A computer-based method of generating a surgical plan comprising: 

reading digital data associated with a 3D (three-dimensional) model of a first patient's 
bone, wherein the digital data resides in a memory in a computer; and 
25 generating a first surgical plan for the first patient's bone based on an analysis of the 
digital data associated with the 3D model. 

12. The method of claim 1 1 , wherein generating the first surgical plan includes specifying an 
osteotomy site on the first patient's bone based on the analysis of the digital data. 

30 
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13. The method of claim 1 1, wherein generating the first surgical plan includes: 
determining whether a second surgical plan exists for a second patient's bone having bone 

contours similar to the first patient's bone; and 
generating the first surgical plan based on information contained in the second surgical 
plan. 

14. The method of claim 13 wherein determining whether the second surgical plan exists 
includes accessing a surgical plan database to read past surgical data therefrom. 

15. The method of claim 11, wherein generating the first surgical plan includes: 
selecting a fixator to be mounted on the first patient's bone; and 

determining a location on the first patient's bone where the fixator is to be mounted. 

16. The method of claim 15, wherein selecting the fixator includes accessing a database to 
choose the fixator from a plurality of fixator configurations stored therein. 

1 7. The method of claim 1 1, wherein generating the first surgical plan includes selecting one 
or more surgical tools appropriate to perform an orthopedic surgery on the first patient's 
bone. 



18. The method of claim 17, wherein selecting said one or more surgical tools includes 
accessing a database to choose said one or more surgical tools from a plurality of surgical 
tool configurations stored therein. 

19. The method of claim 11, wherein generating the first surgical plan includes: 
determining whether a bone distraction is needed for the first patient's bone; and 
preparing a day-by-day length adjustment schedule for each strut in a fixator to be 

mounted on the first patient's bone. 
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20. The method of claim 19, further comprising: 

receiving a plurality of post-surgery X-ray images of the first patient's bone along with 

the fixator mounted thereon; and 
revising the day-by-day length adjustment schedule based on the plurality of post-surgery 

X-ray images. 

21 . The method of claim 11, further comprising: 
displaying the first surgical plan on a computer screen; and 

allowing a surgeon to print one or more portions of the first surgical plan displayed on the 
computer screen. 

22. The method of claim 11, further comprising: 

creating a 3D simulation of one or more surgical procedures contained in the first surgical 
plan; and 

displaying the 3D simulation on a computer screen accessible to a surgeon operating on 
the first patient's bone. 

23. The method of claim 12, wherein generating the first surgical plan further includes: 
determining accessibility of the osteotomy site during an actual orthopedic surgery; and 
revising the first surgical plan based on the accessibility determination so as to generate a 

revised first surgical plan that allows a surgeon to more easily access the osteotomy 
site during the actual orthopedic surgery. 

24. A computer-readable storage medium having stored thereon instructions, which, when 
executed by a processor, causes the processor to perform the following: 

define a 3D (three-dimensional) template model for a patient's bone; 
store a plurality of digitized 2D (two-dimensional) X-ray images of the patient's bone in a 
computer memory; 

extract 2D fiducial geometry of the patient's bone from each of said plurality of digitized 
2D X-ray images; and 
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deform the 3D template model using the 2D fiducial geometry of the patient's bone so as 
to minimize an error between contours of the patient's bone and those of the deformed 
3D template model. 

5 25 . The computer-readable storage medium of claim 24, which, when executed by the 
processor, causes the processor to further perform the following: 
read digital data associated with the deformed 3D template model of the patient's bone, 

wherein the digital data resides in the computer memory; and 
generate a surgical plan for the patient's bone based on an analysis of the digital data 
10 associated with the deformed 3D template model. 

26. A method of providing assistance in planning an orthopedic surgery comprising: 
receiving from a remotely-located surgeon a plurality of 2D (two-dimensional) X-ray 

images of a patient's bone over a communication network; 
15 generating a 3D (three-dimensional) simulation of one or more surgical procedures to be 
performed on the patient's bone, wherein the 3D simulation is generated based on data 
contained in the plurality of 2D X-ray images; and 
sending the 3D simulation along with a surgical plan for the patient's bone to the 
remotely-located surgeon over the communication network. 

20 

27. The method of claim 26, further comprising at least one of the following: 

charging a per-surgery fee to the remotely-located surgeon; 

charging a renewable subscription fee to the remotely-located surgeon; and 

charging a one-time subscription fee to the remotely-located surgeon. 

25 

28. The method of claim 26, wherein the communication network is the Internet. 

29. A system for providing assistance in planning an orthopedic surgery comprising: 

a target computer connected to a communication network, wherein the target computer is 
30 accessible by a surgeon to be operating on a patient's bone, and wherein the target 
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computer is configured to transmit a plurality of digitized 2D (two-dimensional) X-ray 
images of the patient's bone over the communication network; and 
a host computer in communication with the target computer and connected to the 
communication network, wherein the host computer is configured to perform the 
5 following: 

receive over the communication network the plurality of digitized 2D X-ray images of 

the patient's bone sent by the target computer, 
generate a 3D (three-dimensional) simulation of one or more surgical procedures to be 
performed by the surgeon on the patient's bone, wherein the 3D simulation is 
10 generated based on data contained in the plurality of digitized 2D X-ray images, 

and 

transmit the 3D simulation along with a surgical plan for the patient's bone to the 
target computer over the communication network. 

15 30. The system of claim 29, wherein the communication network is the Internet. 
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